Since Scale Invariant Local Ternary Pattern (SILTP) background modeling algorithm is of high complexity and slow computing speed, which is not suitable for real-time video processing, a new method named Uniform Scale Invariant Local Ternary Pattern (USILTP) background modeling algorithm was proposed. Firstly, the feature of USILTP was extracted by regulating the frequency of SILTP coding jump in order to reduce the feature dimension of SILTP. Secondly, a USILTP background modeling parallel algorithm based on Intel core graphics (Intel HD) and Open Computing Language technology (OpenCL) was designed and implemented to further accelerate USILTP background modeling algorithm. Finally, the foreground result of USILTP background modeling algorithm was optimized by combing multiple color channel models. The experimental result shows that the proposed algorithm can be applied to process 320×240 resolution video at a rate of 98 frame/s on the Intel HD 4600, which is 4 times faster than that of SILTP background modeling algorithm. In terms of foreground detection, the performance of the proposed algorithm is improved by 2.1% compared with SILTP background modeling algorithm on the public dataset.